When computation procedures are to be executed by a system that requires data stored in a database, such as a foreign database, the data is typically first read into a process space of the system before the system performs any requested computation(s). Though computation in a system has been continually improved to increase performance, the act of reading data (e.g., from foreign databases) into the system places a great overall time constraint in performing a requested operation.
When table sizes involved in computational programming become large, e.g., in the hundreds of millions or even billions of observations, the performance of data fetching, as measured in response time as well as needed disk (or memory) space to store the data, into an analysis session to execute a procedure may be very expensive or resource prohibitive. Moreover, in many instances, the computational results of the analysis procedure may be stored back into a table located on a foreign database. The act of writing the data back to the database can also be expensive when the results from the computation are large.
In conventional systems, to execute a procedure, the data is read from the database into an analysis session, where computation is performed, and results of the computation in many cases written back to the database from which the data was retrieved. This procedure is known as an “outside-the-database” model. To speed up the response time, many procedures have been architected to operate using multiple operating system (OS) threads, using symmetric multi-processing (SMP) techniques. However, the performance of SMP techniques does not eliminate or reduce the amount of data that must be transmitted across a network between a client application that performs the computation and the foreign database.
It is with respect to these and other considerations that the present improvements have been needed.